This work explores how to fine-tune large language models using prompt engineering techniques with contextual information for generating an accurate text description of the full story, ready to be forwarded to off-the-shelve speech synthesis tools. We propose to use existing computer vision and optical character recognition techniques to build a grounded context from the comic strip image content, such as panels, characters, text, reading order and the association of bubbles and characters. Then we infer character identification and generate comic book script with context-aware panel description including character's appearance, posture, mood, dialogues etc. We believe that such enriched content description can be easily used to produce audiobook and eBook with various voices for characters, captions and playing sound effects. Keywords: comics understanding large language model prompt engineering character identification comic book script accessible comics.

Augmented Language Models (ALMs) blend the reasoning capabilities of Large Language Models (LLMs) with tools that allow for knowledge retrieval and action execution. Existing ALM systems trigger LLM thought processes while pulling observations from these tools in an interleaved fashion. Specifically, an LLM reasons to call an external tool, gets halted to fetch the tool's response, and then decides the next action based on all preceding response tokens. Such a paradigm, though straightforward and easy to implement, often leads to huge computation complexity from redundant prompts and repeated execution. This study addresses such challenges for the first time, proposing a modular paradigm ReWOO (Reasoning WithOut Observation) that detaches the reasoning process from external observations, thus significantly reducing token consumption. Comprehensive evaluations across six public NLP benchmarks and a curated dataset reveal consistent performance enhancements with our proposed methodology. Notably, ReWOO achieves 5x token efficiency and 4% accuracy improvement on HotpotQA, a multi-step reasoning benchmark. Furthermore, ReWOO demonstrates robustness under tool-failure scenarios. Beyond prompt efficiency, decoupling parametric modules from non-parametric tool calls enables instruction fine-tuning to offload LLMs into smaller language models, thus substantially reducing model parameters. Our illustrative work offloads reasoning ability from 175B GPT3.5 into 7B LLaMA, demonstrating the significant potential for truly efficient and scalable ALM systems.

The hype around artificial intelligence has been building for years, and you could say it reached a crescendo with OpenAI's recent release of ChatGPT (and now GPT-4). It only took two months for ChatGPT to reach 100 million users, making it the fastest-growing consumer application in history (it took Instagram two and a half years to gain the same user base, and TikTok nine months). In Ian Beacraft's opinion, we're in an AI hype bubble, way above the top of the peak of inflated expectations on the Gartner Hype Cycle. But it may be justified, because the AI tools we're seeing really do have the power to overhaul the way we work, learn, and create value. Beacraft is the founder of the strategic foresight agency Signal & Cipher and co-owner of a production studio that designs virtual worlds. In a talk at South by Southwest last week, he shared his predictions of how AI will shape society in the years and decades to come.

AI art

Even though it is possible to make a comic book yourself, only some people are good at it. Writing comics is hard and takes a lot of work. Most writers create a story and then work with other artists to put it on paper. This was the case until yesterday. Today, the rules have changed!

On Monday, Heritage Auctions sold off an unopened copy of the 1996 Nintendo 64 game Super Mario 64 for $1.56 million. According to the auction house, there were 16 bids, and the final price is the highest ever for a single video game. The vintage video-game trade has been booming over the past three years, especially during the pandemic, but this selling price even has seasoned collectors shocked and scratching their heads. A sealed Super Mario 64 just sold for $1,560,000 @HeritageAuction. "I myself and many other in this space are just blown away with these results," said Donald Brock Jr., owner of the collectibles site Columbia Comics.

Welcome to the second of our monthly digests, designed to keep you up-to-date with the happenings in the AI world. You can catch up with any AIhub stories you may have missed, get the low-down on recent conferences, and generally immerse yourself in all things AI. You may be aware that we are running a focus series on the UN sustainable development goals (SDG). Each month we tackle a different SDG and cover some of the AI research linked to that particular goal. In February it was the turn of climate action.

Mayra Leiva of Reseda, California, knew her eight-year-old son was a little interested in history. But she was surprised when all at once he became a walking encyclopedia, spouting dates and pretending every tire swing was a time machine. "It happened after he saw Night at the Museum," she says. I've had to do a lot of Googling to keep up!" Not many children will tell you that their favorite school subject is history. Memorizing dates and learning long-ago facts that don't seem relevant isn't exactly high on their fun list. Perhaps that's why pop culture--movies, music, television, and even video games and comic books--can be such useful teaching tools. "Teaching through pop culture helps students relate history to their own background and experiences," says Gail Hudson, a fifth-grade teacher and 2020 Nevada Teacher of the Year. "It's tying into something that's already caught their interest." Take the movie version of the Broadway show Hamilton, which releases on Disney July 3.

WASHINGTON – Once confined to comic books, exosuits that enhance a wearer's physical abilities took a step forward Thursday as researchers unveiled a pair of robotic shorts that assist in walking and running. The entire get-up, which includes a battery that straps around the waist and a motor on the lower back that connects to pull-cables, weighs just 5 kilograms (11 pounds) and detects its wearer's gait to appropriately adjust its output. Walking and running are very different activities from a biomechanical viewpoint, and previous devices had focused on boosting one or the other, but not both, co-author Conor Walsh from Harvard's Wyss Institute for Biologically Inspired Engineering said. "So I think it's a step towards these devices not only helping with a single activity, but devices that eventually can help people in their everyday lives, in many different ways across many different activities," he said. The breakthrough required developing a control algorithm that used three sensors to detect with 99 percent accuracy what the wearer was doing and respond accordingly.

Early on, fans had every reason to be wary of Sony's Oscar-winning "Spider-Man: Into the Spider-Verse." The studio's last three Spider-movies outside the Marvel Cinematic Universe had seemed trapped in a web of confusion. And wouldn't a new release by the studio -- even an animated one -- conflict with the extremely popular Tom Holland iteration in the MCU? But all "Spider-Verse" has done since its release is earn the best reviews of any Spider-movie, ever (97% on Rotten Tomatoes) and sweep awards-season honors, including the animated feature Academy Award on Sunday night. It bested two competitors from Marvel's parent company, Disney, including early favorite "Incredibles 2" and "Ralph Breaks the Internet," as well as the smaller-scale "Isle of Dogs" and Japanese import "Mirai." Co-producer Chris Miller accepted the Oscar by saying, "There's 800 filmmakers [on this film] who pushed boundaries and took risks to make people feel powerful and seen."